Applying DEA Technique to Library Evaluation in Academic Research Libraries

INCREASINGLY, their use of resources IBRARIES A R E ASKED T O JUSTIFY in terms of producing meaningful services and impacts to the users and the parent organizations. This study applied an analytical technique called Data Envelopment Analysis (DEA) to calculate the relative technical efficiency of ninety-five academic research libraries that are members of the Association of Research Libraries. Instead of providing the average performance among libraries, DEA, with the proper model of library inputs and outputs, can reveal the best practices in the peer groups, as well as the technical efficiency score for each library. The technique was applied to the libraries using the 1996 and 1997 ARL annual statistics. The study also reviews the applications of DEA technique in the library environment. INTRODUCTION Researchers recognize two broad aspects of evaluating library performance: “effectiveness” and “efficiency.” Effectiveness here means the extent to which library services meet the expectations or goals set by the organization. In the library field, there has been a growing desire to measure effectiveness in terms of impact of library services on their users. The second aspect of library performance measurement, “efficiency,” measures the library’s ability to transform its inputs (resources) into production of outputs (services), or to produce a given level of outputs with the minimum amount of inputs. The efficiency aspect of library performance has received less attention in the library literature, but it is an immediate concern for decision-makers at the parent institution. The success of the library, like that of other organizations, depends on Wonsik Shim, School of Information Studies, Florida State University, 101 Shores Building, Tallahassee FL 32306-2100 LIBRARY TRENDS, Vol. 51, No. 3, Winter 2003, pp. 312-332 02003 The Board of Trustees, University of Illinois SHIM/DEA TECHNIQUE IN LIBRARY EVALUATION 313 its ability to behave both effectively and efficiently. We can put these two dimensions of library performance in a 2 by 2 matrix as shown in Figure 1. Performance improvement requires constant and careful monitoring and assessment of library activities and operating environments. This, in turn, requires the development of proper measurement tools or devices. This study assesses the technical efficiency of academic research libraries that are members of the Association of Research Libraries using a complex tool called DEA. While the development of effectiveness is equally important, this study is focused solely on measuring library efficiency. Fzgure 1. Library Performance Matrix Using the Levels of Effectiveness and Efficiency as Two Dimensions. x z 6. Effective but Excessively Costly Best All-around Performers T .9 m fE U w Efficiently Managed for ? Problematic, Underperforming Insignificant Results 4 4 DATAENVELOPMENT ANALYSIS ovmim Data Envelopment Analysis (DEA) measures the relative efficiencies of organizations with multiple inputs and multiple outputs (Charnes et al., 1978). The individual organizations, teams, or units analyzed are called the decision-making units, or DMUs. The basic point of DEA is to identify the so-called efficient frontier in some comparison set of DMUs. All units on this frontier are said to be operating at 100 percent efficiency. DEA provides an efficiency score for each of the inefficient units, as well as a benchmark set of efficient units that lead to that conclusion. The results of the DEA analysis can be used in performance measurement of libraries, especially for benchmarking purposes. Since the DEA technique was first developed by Charnes, Cooper, and Rhodes in 1978, it has been widely applied to industries as diverse as health care, finance, education, and transportation, as well as many other industries and organizations. The technique is well documented in both the operations research (Banker, Charnes, & Cooper, 1984; Dyson & Thanassoulis, 1988; Golany & Roll, 1989; Cooper, Thompson, & Thrall, 1996) and economics literature (Sengupta, 1987; Banker & Maindiratta, 1988; Seiford & Thrall, 1990; Leibenstein & Maital, 1992). The DEA bibliography compiled by Seiford (1994) includes more than 400 articles, books, and disser-314 LIBRARY TRENDS/WINTER ZOO3 tations between 1978 and 1992. Arecent bibliography (Emrouznejad, 2001) reports more than 1,000 applications of the DEA technique. DEA allows the weights of individual inputs and outputs of each DMU to vary until it gives the best possible combination for the focus library. In DEA calculations, through mathematical optimization, each DMU is assigned the weights that maximize its efficiency score. In doing so, DEA gives all the other DMUs “the benefit of the doubt” by allowing them to apply the same weights to see if any of them looks better than the library being evaluated, which is called the “focus” DMU. If the focus DMU looks at least as good as any other DMU, it receives an efficiency score of 1. However, if some other DMU looks better than the focus DMU, even when the weights are calculated in a way that is most favorable to the focus, it will receive an efficiency score less than 1.In DEA, a separate calculation is done for each DMU. Graphical Illustration Supposr, for the sake of illustration, we have seven libraries or DMUs that each have only one input and output. We assign these libraries to the coordinate values associated with the points L1 through L7 in Figure 2 where the input is represented on the horizontal axis (X) and the output is represented along the vertical axis (Y). Figure 2. Envelopment Surface. Adapted from Charnes et al. (1994),p. 33.